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Abstract 

Turbo codes are a very efficient method for communicating reliably through 
a noisy channel. There is no theoretical understanding of their effectiveness. In 
Ref. m they are mapped onto a class of disordered spin models. The analytical 
calculations concerning these models are reported here. We prove the existence of 
a no-error phase and compute its local stability threshold. As a byproduct, we gain 
some insight into the dynamics of the decoding algorithm. 
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1 Introduction. 



Communication through a noisy channel is a central problem in Information Theory 
||. Error correcting codes are a widespread method for compensating the information 
corruption due to the noise, by cleverly increasing the redundancy of the message. Turbo 
codes P, U are a recently invented class of error correcting codes with nearly optimal 
performances. They allows reliable communication (i.e. very low error per bit probability) 
with practical communication rates. 

It is known, since the work of Sourlas || [7|, |J, that there exists a close relationship 
between the statistical behavior of error correcting codes and the physics of some disor- 
dered spin models. Recently the tools developed in statistical physics have been employed 



in studying Gallager-type codes [11, [Hi 12 



In Ref. |I| the equivalence discovered by Sourlas is extended to turbo codes, and the 
basic features of the corresponding spin models are outlined. A remarkable property of a 
large family of turbo codes, presented in Ref. [TJ, is the existence of a no-error phase. In 
other words the error probability per bit vanishes beyond some critical (finite) signal to 
noise ratio. In Ref. [ l] some intuitive arguments supporting this thesis are given. Some 
analytical results concerning the critical value of the signal to noise ratio are announced 
without giving any derivation. These results are compared with numerical simulations. 

In this paper we present the analytical results in their full generality, and explain their 
derivation. We prove the existence of the no-error phase and find the condition for its 
local stability. This condition is derived in two different approaches. In the first one we 
study the asymptotic dynamics of the decoding algorithm. In the second approach we use 
replicas and establish the condition for stability in the full replica space. Local stability 
is a necessary but not sufficient condition for the stability of the no-error phase. The 
critical signal to noise ratio obtained from local stability is the correct one only if the 
phase transition is a second order one: in the general case it is only a lower bound. 

The spin models which are equivalent to turbo codes have the following statistical 
weight 

i N 

N N 

ffW(cr) = -^jfe^-^fW) (1-2) 

i=\ i=l 

The dynamical variables of the model are the spins er^ = {a[ , . . . , crff} with k = 
1,2. We shall choose them to be Ising spins [], that is a!- = ±1. The spins enters in 
the hamiltonians H^ k \a) through the local interaction terms ej(cr) and rji{cr) which are 
products of cr's. Their exact form can be encoded in two set of numbers k(J; 1) = 0, 1 

and k(J; 2) = 0, 1 as follows: e;(<x) = Yl r j=o a i-j anc ^ 7 li( <T ) = YYj=o a i-j 2 ^ ■ m order to fix 
completely our notation we set k(0; 1) = k(0; 2) = 1. The quenched variables are: 



lr rhis corresponds to considering codes which works with a binary alphabet. 
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• the couplings J = {jf^/if }, whose distribution V(J) = Ui, k p ( J i k) ) P(h[ k) ) 
satisfies the conditions fdJ$ k) P(4 fe) ) jf } > and Jdhf ] P(h\ k) j hf ] > 0; 

• the permutation p : {1, . . . , N} — > {1, . . . , A^}, which has uniform distribution. 

It is convenient to impose a fixed boundary condition at one end of the chain (i.e. Oi = +1 
for i < 0) and a free boundary condition at the other end. The model is composed by 
two one dimensional substructures (chains), which interact through the Kronecker delta 
functions in Eq. ( |1 . 1|) . When the average over permutations is taken into account this 
interaction turns into a mean field one. This interplay between the two subsystems, each 
one possessing a one dimensional structure, and the mean field interaction which couples 
them is clearly displayed by the analytical calculations. For further explanations on Eqs. 
1.1- and their motivation we refer to 0. 



The paper is organised as follows. In Sections |2| and |3] we present a first derivation of 
the stability condition. We write a "mean field" equation which describes the dynamics 
of the decoding algorithm (Sec. |2|), we show that it possesses a no-error fixed point and 
then study its behavior in a neighbourhood of this fixed point (Sec. |3|). Thanks to this 
derivation we will understand how this fixed point is reached. In Section f| replicas are 
introduced in order to compute the average over the permutations. We exhibit the no- 
error saddle point. In Section |^ the stability of the no-error saddle point is studied by 
diagonalizing the second derivative of the free energy. Finally in Section ^| the validity 
of our calculations is discussed. Appendix [A| collects some useful (although simple) facts 
of algebra. In Appendix |B| the type of integral equations which appear in Section |3] is 
studied in detail. 



A "mean field" equation for the decoding algo- 
rithm. 



Some properties concerning the models defined by Eqs. ( |1.1J - |1.2| ) can be obtained by 
considering the "turbo decoding" algorithm and making some factorization hypothesis. 
These hypothesis enable us to obtain a recursive integral equation for the probability 
distribution of a local field. They can be justified on heuristic grounds and arguments of 
this kind will be given later in this Section. Moreover the replica calculation presented in 
the Section ^ does support our arguments. In particular this approach allows us to derive 
the critical noise below which "perfect" decoding is possible. 

Turbo decoding is an iterative algorithm. The iteration variables are the fields T^ k ' = 
{rf^, . . . , r£ } with k = 1, 2. The step t of the turbo decoding algorithm is defined as 
follows HI: 



lfty + 1) 

vf\t + i) 



— arctanh 

P 

—arctanh 

P 



(e p - 1 (i)( cr ))r 2 ( ) 2) 



(6 p(i) (ar))« )(t) J -r^(t) 



(*) 



(2.1) 
(2.2) 
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The expectation value is intended to be taken with respect to the Boltzmann 

weight with the modified hamiltonian H^ k \cr) — YliLi F^e^tr). The iteration variables 
T^' should be interpreted as external fields conjugate to the operators e^cr^). They 
describe, in an approximate way, the action of each of the two chains on the other one. 
In order to lighten the notation, let us write Eqs. (|2.1[ - |2.2| ) in the form: 



r (fc) (t + 1) = F p (fc) (r (fc,) (t), j (fe,) ) (2.3) 

with k' — 2 if k — 1, and k' — 1 if k — 2. Due to the randomness in the couplings J, 
the fields T are random variables. Equation ( |2.3D implies an integral equation for the 
probability distribution of T: 

v t+1 (rM) = J drw j dj^v t {v {k '\ jW)5 [r« - Ff (r^'\ j^] (2.4) 



Let us state a few approximations which allow us to reduce Eq. (|2.4|) to a much simpler 
one. 

(1) We make the substitution V t (T {k '\ J (fc,) ) -> P t (r (fc,) )P( J (fc ' } ) in Eq. 0. This 
yields a closed integral equation describing the evolution of the distribution Vt(T^). 

(2) We neglect correlations between the fields at different sites: 

N 

V t (T^)^H4f(T^) (2.5) 

i=i 

These two hypothesis imply that Eq. ( [2.4|) is equivalent to: 

^%i(y) = J dir { t k) [x} J dV[J] 6 (y - -^arctanh ((e^er)) j x ) + x fj{i ^j (2.6) 



N 



dnP[x] = Y\dx l Ti { t'\x i ) (2.7) 

i=l 

where p is the appropriate permutation of {1, ... , N}, i.e. p = p _1 if k = 1 and p = p if 
k = 2. The expectation value {■) j x on the right hand side of Eq. (|2.6| ) has to be taken 

with respect to the hamiltonian H(cr) = — Y^i=x(^i + x i) e i{< J ) ~ J2iLi hirji(er). 

Let us now define a field distribution averaged over the permutations and the sites: 

•"w-iSsE'S'wrt (2-8) 

p i=l 

(k) 

where we made explicit the dependence of 7r^ t upon the specific permutation p which 
defines the code. We can now state our last approximation. 

(3) We make the substitution 7r^(x|p) — > n[ k \x) on the right hand side of Eq. (|2.6|). 
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xt (2.9) 



This yields a recursive equation for . 

f*+i(y) = JjYlj d7lt ^ J dV \ J \ 5 [v- ^arctanh (^(cr)) j X J + 

The indices (k) and (k') have been dropped since we can define ir t = 7r t for t odd, and 
n t = nf' for i even, or vice- versa. A byproduct of this heuristic derivation is the expression 
for the probability distribution of the expectation values (ej(er)) after t iterations of the 

turbo decoding algorithm: V t (e) = ^ J2iLi f-^ dir t [x] f dV[J] 5 (e - (e»(<r)) j x j . 
Let us discuss the validity of the approximations made in deriving Eq. (|2.9|) . 

(1) and (2) These approximations should be accurate in the thermodynamic limit for a generic 
random permutation p. The reason is that the correlations produced by Eqs. fl2.1| - 
|2.2| ) have short range: (ej(er)) and (e,(cr)) have a significant correlation only if 
is less than some characteristic length. The random permutation p reshuffles the 
sites so that the correlation between two fields and Tj is vanishing with high 
probability if \i — j\ is required to be "small". The correlations which "survive" 
(non vanishing only between "distant" sites) are irrelevant when computing the 
expectation values of local operators. In order to make this last assertion plausible, 
let us suppose that, for each site i, we can find a "large" [] interval [i—L(N),i+L(N)] 
of the chain, such that the correlations between the couplings inside the interval are 
negligible. The expectation value (£i(cr))j x will not depend (as iV — > oo) upon the 
couplings outside [i — L(N),i + L(N)} (this is always true in one dimension at non 
zero temperature) and can be then safely computed without taking into account 
the correlations. It is easy to find a similar argument concerning the correlations 
between r (fc,) and J (fc,) in Eq. 



(3) This is the probabilistic analogue of the replica symmetric approximation. Let us 
consider the fixed point equation n t +i = ir t corresponding to the dynamics defined 
by Eq. (|2.9| ). It is remarkable that this fixed point equation coincides with the 
saddle point equation obtained by the standard replica method in the replica sym- 
metric approximation (see Section |j). This fact confirms our conclusions about the 
relevance of the various approximations. 



3 The behavior of the decoding algorithm. 

Equation (|2.9|) is the final outcome of our heuristic derivation. We want to study its 
behavior when the distribution tt(x) is concentrated on large values of the field x, that 
is when the error probability is very small. In this regime the most relevant spin config- 
uration satisfies ej(cr) = +1 for each % = 1, . . . , N. The lowest excitations are such that 
e.j(cr) = — 1 only on a few sites. The first crucial point will be to understand that, for a 
class of hamiltonians of the type (|1.2|) (which will be defined as "recursive "), the energy 

2 Here "large" means that linijv->oo L(N) = oo. 
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to be paid for flipping a single e variable diverges in the thermodynamic limit. The second 
point will be to evaluate the energy to be paid for flipping two e variables. In order to 
treat both these passages in full generality it is convenient to use an algebraic bookkeeping 
technique which we shall soon explain. The results concerning these two points will be 
useful again in Section ||. 

A preliminary step consists in making the change of variables Xi = e~ 2l3xi and intro- 
ducing the corresponding distribution function Q t (X)dX = ir t (x)dx. Low X's correspond 
then to large local fields, i.e. to low error probability. The result is 

Qt+1{Y) -L±[ dQAX] j d v W s (r -^ ggl^g ) (3.D 

where 

N 

Z(e i (<r) = e;J,X) = Z i (e) = £ e -W) J] xf'^ (3.2) 

(T:ti{(T)=t k=l 

with H(<t) = —J2i^i e i( (T ) ~ J2i hiVii* 7 ) ■ Let us introduce some notations in order to 
write down the small X expansion of Zi(e): (ki,...,ki) is an Z-uple (not ordered) of 
integers in {1, . . . , i — 1, i + 1, . . . , N}; cr is the configuration such that ej(cr) = +1 for all 
the sites i; cr(k, l,m, . . .) is the configuration such that €j{cr) = —1 if j — k,l,m, . . . and 
6j(cr) = 1 otherwise (there is at most one such configuration once the boundary conditions 
have been specified); E = H(er ) is the energy of the ordered configuration; finally 
A(k, l,m, . . .) = H(er(k, l,m, . . .))— H(<tq). The following expressions are straightforward: 

N-l 

Z i (+l) = e-f ,E °Y, E X kl ...X h e-^-^ (3.3) 

1=0 (fei,...,fcj) 
N-l 

Z i (-l)=X i e-f us °Y, E X kl . . . X kl e-^ k ^ (3.4) 

(=0 (fci,...,fei) 

The "bookkeeping technique " which we shall adopt in treating the above expansions 
consists in using the algebra of "generating polynomials" [0] . This approach allows us to 
consider a general hamiltonian of the type (|1.2| ). Let us define the following polynomials 
on Z 2 : G(x) = YlT=x G j xj i with °i = 9n(x) = Ej=o n ) xj '' S {n) (x) = g n (x) ■ 

G(x) = Yl'jLi Gj Notice that the boundary condition on cr can be translated as 

follows: G(x) is a series of strictly positive powers of x. 

It is necessary to distinguish two types of models: in the first case gi(x) divides g2{x), 
i.e. g2(x)/gi(x) is a polynomial (these are the "non recursive" models, a particular case 
being ej(cr) = dj); in the second one g±(x) does not divide (72(^)5 i-e. g2(x)/gi(x) is a 
series ("recursive" models). 

We shall treat the "recursive" models first. In this case the first order terms in the 
expansions ( |3.3p and ( |3.4| ) are exponentially small in the size. In order to prove this 
assertion, let us consider the configuration cr(l). The relevant generating polynomials are 
Q^-'{x) = x l and Q^\x) = x l g 2 (x) / gi(x) . The form of Q^^x) is given by the following 
result of algebra 
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Lemma 3.1 Let g(x) and f(x) be two polynomials on Z2 such that g(0) = f(0) = 1, 
f(x) ^ 1, and their greatest common divisor gcd(f(x), g(x)) is equal to 1. Then there 
exists an integer uj such that g(x)/ f(x) = Y^=o xnul Pn( x ) with deg[p n (x)} < uj andp n [x) = 
Poo{%) 7^ if n is large enough. Hereafter we shall callu(f) the smallest of such integers. 

An explicit expression for 00(f) is given in the Appendix |Aj. The Lemma |3.1| applies 
to our case if we divide both g±(x) and 02 (x) by their greater common divisor: fk(x) = 
9k(x)/ gcd((7i(x), g2{x)), so that gcd(/i(x), f2(x)) = 1. It implies that if we write down the 
numbers r]j(cr(i)) = ±1 we get an antiperiod followed by a non trivial periodic sequence 
with period uj(fi)- Let us consider a site "in the bulk": N5 < i < N(l — 5) with 5 a 
(small) positive number. Then, using the convention hj = for j > N, we get: 

N 00 1 

A(i) = 2J; + 2^£j% = 2J; + 2^ Pn,kh i+n ^ h)+k (3.5) 

j=l n=0 k=Q 

which diverges almost surely in the thermodynamic limit if (h) > (see the Introduction 
on this point). In Eq. ( |3.1|) we must sum also terms which are "near" the boundaries, i.e. 
i < N5 or % > N(l — 5). These give however a negligible contribution. 

Let us now consider the second order terms of the expansions ( |3.3| ) and ( |3.4|) . They 
involve configurations cr(k,l) with two flipped e(<r)'s. The only configurations which 
give a non negligible contribution are the ones which involve a finite (in the — > 00 
limit) number of flipped ?](cr)'s. This corresponds to choosing k and / such that (x h + 
x l ) g2{x) / gi(x) is a polynomial (and not an infinite series). The following useful result is 
proved in the Appendix 0. 

Lemma 3.2 Let f(x) be a polynomial on Z2 such that /(0) = 1 and k an integer. Then 
there exists an integer ou(f) such that f(x) divides 1 + x k if and only if k is a strictly 
positive multiple of ui{f). 



As suggested by the notation the a;(/)'s cited in Lemmas |3J] and [T2] are indeed equal. 
The terms which give a non vanishing contribution at order X 2 in the expansions (|3.3| - |3.4| ) 
are the ones corresponding to configurations cr(k, I) such that \k — 1\ is a multiple of tf(/i). 
In order to evaluate these terms we must count the number of flipped ^(cr)'s. This number 
is nothing but the number of non zero coefficients in the polynomial {x k + x l )g2(x) /gi(x). 
Let us define the weight of a polynomial p(x) = YlkPk xk over ^2 as the number of its non 
zero coefficients: weight(p) = # {pk\pk 7^ 0}. The weight of (x k + x l )g2(x) / g\(x) is given, 
for a large class of hamiltonians, by the following lemma. 

Lemma 3.3 Let f{x) and g{x) be two polynomial on Z2 such that f(0) = g(0) = 1, 
f(x) ^ 1, and gcd(f(x),g(x)) = 1. If deg(g) < u>(f) then the weight of s m (x) = (1 + 
x muJ (f')g(x)/f(x) is given by weight(s m ) = w (f,g) + W\(f, g)m for each m > 1. The 
coefficients Wo(f,g) and Wi(f,g) are positive integers whose explicit expressions are given 
by Eqs. / fA^ - |X^ j. 

Appendix |A] contains also an illustration of what could happen in the more general case. 
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By using Lemmas |3.2| and 3J3 we can linearize with respect to X the expression on 
the r.h.s. of Eq. (Q: 



1 Zj(-l) 
X % Zi(+1) 



^<+mw(/i) e 



•j9A(i,i+mw(/i)) 



0(X 2 



and defining s m (a;) = (1 + x muj ^)g2(x) / gi(x) = 52,- s m,j x ^ we get 



(3.6) 



A(fc, /) — 2J k + 2 J; + 2 s TOj j/i m i n (fe^) + j 



(3.7) 



if |Jfc-J| = mw(/i). Clearly Eq. Q holds only for z in the "bulk" (i.e. iV5 < z < (1-S)N) 
up to terms which are exponentially small in the size N. 



Our first important observation is that the right hand side of Eq. (|3.6f) vanishes if 
Xk = for k — 1, . . . , N. This means that Q*(X) = S(X) is a fixed point of Eq. ( |3.1| ) 
for "recursive" models. Recall that the change of variables which yields Eq. ( [3 . 1|) is 
X = e -2 ^ and that x has the meaning of an effective field acting on ej(er). The solution 
Q*{X) corresponds then to a phase with completely frozen spins: (ej(cr)) = +1. 

We would like to understand if this phase is stable for some temperature j3 and some 
distribution of the couplings. A possible approach is to study the turbo decoding dynamics 
(as described by Eq. (|3.1|)) when starting from a distribution "near" Q*{X). Let us 
suppose that, for Qt(X) near enough to Q*(X), we can safely neglect 0(X 2 ) terms on 
the r.h.s. of Eq. flglf ): 



Qt+i(Y) 



1 jr HdQAx] I dv[j] s(y-J2 x„ 

i=l V m^O 



(3.8) 



This equation is very similar to a class of recursive equations which appear in a completely 



different context: polymers on disordered trees |T3], [14], [15|, [16|, [17| . These are of the type 



P m (Z) = ^ ^dZ^^Zi) Jp{V l) ... 1 V K )dV 1 ...dV K 5\z-^e-^Z i \ (3.9) 

The only non trivial difference is that the linear function of X appearing inside the delta 
function on the r.h.s. of Eq. ( |3.8| ) depends upon a macroscopic (indeed linear in N) 
number of X's. In Eq. (|3.9| ), instead, only a finite number of variables appears: K 
is the coordination number of the tree minus one. Notice however that, for m large, 
A(i, i + mw(/i)) ~ 2 weight (s m )(h) ~ 2wi(fi, f2) m (h). We can thus truncate the sum in 
Eq. Q3.8p to m < M by making an error of order 0(e~ cM ) and we guess that the limit 
M — > oo can be taken at the end without problems 0. 

Let us summarize some results of [[0| which are useful in our discussion. It turns out 
that Eq. (|3.9|) is equivalent to a discretization of the Kolmogorov-Petrovsky-Piscounov 



3 This argument is not mathematically rigorous since it is not honest to use the central limit theorem 
in this case: we refer to Appendix M for more convincing arguments. 
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(KPP) equation [|18] (a well studied partial differential equation). Using this equivalence 
the large time limit of Eq. ( |3.9|) is obtained: 

P t (X) -> e-MW P{Xe~ mt ) (3.10) 

corresponding to a front wave solution of the KPP equation with front velocity c(/3). If 
we define the function 



rt i) = ^Iom ) V /./I ,..../!,, plT, 1 ■,,),-" | (3.111 

then the front velocity is given by the following construction: 



with (3 C given by 

v{(3)=0 (3.13) 



d 

df3 



At the critical temperature (3 C a freezing phenomenon takes place with the front velocity 
sticking to its minimal value. 

Let us apply these results to our case, i.e. to Eq. ( |3.8|) . The large time solution 
Qt{X) ~ e -P c W l Q(Xe -/3c( ^') gives the correct behavior for t — > oo only if c((3) < 0. In 
this case \im t ^ooQt(X) = Q*(X) and it is then correct to linearize Eq. (|3.1[) : the frozen 
phase is stable. If, on the other hand, c(/3) > then we must take into account higher 
order terms in the low X expansion and the asymptotic form is no longer of the type 
defined by Eq. ( |3.10| ): the frozen phase is unstable. 

In the thermodynamic limit we get 



J2 I dV\J]e-^ i+mu>{ - h)) = (3.14) 
JdJP{J)e-^ J j dhP{h)e^ h 



The front velocity c{0) is obtained by applying the construction given in Eqs. ( |3.12| - |3.l3| ) 
to Eq. ( |3.14j ). If the hypothesis of Lemma p.3| are satisfied we can easily sum the series: 



ft \ 2 / t \ w "(hJ'2)+w 1 (f 1 J 2 ) 

2 I dJ P(J) e- 2 <") dh P(h) e- 2 "") 
^ W 7^ ^uL (3-15) 



dh P(h) e 



-2f3h 



We discuss now Eq. (|3.15|) , the more general case being completely analogous. The series 
converges only if Jdh P{h) e~ 2f3h < 1. If jdh P(h) h > 0, as we supposed sine the 
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beginning, then convergence is assured for < (3 < (3\ with J dh P(h) e~ 2 ^ lh = 1. It is 
easy to see that (3v{(3) is strictly convex for < (3 < (3i and thus v(f3) has either one global 
minimum or is strictly monotonic for < (3 < (3\. Since lim j g_ >0 + V {P) — hnig^- v(/3) = 
+oo the first possibility is excluded and we conclude that < /3 C < fa. The important 
point is that the right hand side of Eq. (|3.14j ) is well defined every time we need of it, i.e. 



for < (3 < (3 C . 

In applications to turbo codes a simplification occurs: we are interested in a particular 
temperature, (3=1, and we are left with a unique parameter: the signal to noise ratio 
1/w 2 . Moreover the probability distributions of the couplings are fixed by the character- 
istics of the communication channel || |l| . If we introduce the auxiliary variables J and h, 
which correspond to the output of the channel, the probability distributions are obtained 
as follows 

P(J) dJ = P(J\ + 1) dJ with J = -log P ^ + 1 ) (3.16) 
1 ^ 1 ' ^ 2 S P(J|-1) 1 ^ 

where P{J\r) is the probability distribution of the output of the channel conditional 
to the input r. A similar expression holds for h. If the channel is symmetric (i.e. if 
P(J\ — 1) = P(—J\ + 1)) one easily obtains fa = 1 and then c(f3 = l,w 2 ) = v((3 c ,w 2 ). 
We can distinguish the two cases defined below. 

• If v(f3,w 2 ) < for some < (3 < 1 then we are in the no-error phase and the 
turbo decoding algorithm converges to the message with "velocity" c(/3 = l,w 2 ) = 
min </3<i v(P, w 2 ). We expect the condition v(j3 c , w 2 ) < to be verified in the "low 
noise" region w 2 < wf oc . 

• If v(/3, w 2 ) > in the interval < (3 < 1 then c(/3 = 1, w 2 ) > and the linearization 
in Eq. ( |3.8|) is no longer reliable. In this case 7Tt(x) is expected to converge for 



t — » oo to some distribution supported on finite fields x. The decoded message will 
be plagued by a finite error probability per bit, no matter how many times do we 
iterate the turbo decoding algorithm. 

Let us now study some examples. We consider a gaussian channel with: 

P{J\r) = 77-4t7^x P { — -y- } (3. 17) 



(Anw 2 ) 1 / 2 




(3.18) 



This choice of the variances is justified since it corresponds to a code with rate 1/3 (see 
Ref. fll|). It is useful to define the function 



2/3J 



\ 2 


[2^-1)1 


1 = exp 


w 2 



(3.19) 



The three cases below have been already considered in Ref. (jT]. We refer to the Appendix 
[A] for the calculation of the constants w and u>i to be used in Eq. ( |3.15|) . 
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(a) . A model with nearest neighbours interaction is: Cj(cr) = c^ovi and f]i{cr) = G{ 

(which corresponds to the generating polynomials g\[x) = 1 + x and gi{x) = 1). 
Using Eq. ( |3.15| ) and the fact that w (fi, / 2 ) = and / 2 ) = 1 we get 

M = h og 2£^f}- (3.20) 

jj 1 — z{p, W z ) 

It is easy to see that w 2 ) > for each < (3 < 1 if w 2 > w 2 oc = 1/ log 4. 

(b) . For 6j(cr) = o"jO" i _icr i _ 2 and ^(er) = crj<7j_ 2 (generating polynomials: <7i(x) = l + x + 

x 2 and <? 2 (x) = 1 + x 2 ) we obtain w (fi, / 2 ) = 2 and / 2 ) = 2 and then 

2 1 2z 5 Q3,w 2 ) 

^' w) = ^ log T^W) (3 - 21) 

Finally io 2 ^ = —1/(2 log£ c ) where z c is the only real solution of the equation 2z 5 + 
z 2 = l. 

(c) . If we consider the model given by ej(cr) = OiOi^xOi-i^i-^i-^ and r/i(cr) = aia^i 

(generating polynomials: gi{x) = 1 + x + x 2 + x 3 + x 4 and g 2 (x) = 1 + x 4 ) we obtain 
wo(/i,/2) = 2 and wi(/i,/ 2 ) = 2 as in the previous example. Both v((3, w 2 ) and 
w ; 2 oc coincide with the ones obtained above. 

Let us make a few observations about the validity of our calculation. The threshold 
wf oc has been obtained by starting from a distribution Q{X) very near to the "frozen" one 
Q*(X) and linearizing Eq. (|3.1|) in X. It must then be interpreted as a threshold for local 
stability of the "frozen" solution. Moreover, if we take seriously the heuristic derivation 
of Eq. (|2~9l ), we can deduce something about the dynamics of the turbo decoding algo- 
rithm in the error- free phase: the probability distribution of the auxiliary fields r[ k \t) 
moves towards infinitely large fields with an average velocity c((3,w 2 ). This conclusion 
is compared with numerical data in Fig. ([]]): the agreement seems to be quite good. 
An interesting outcome of the previous calculation is that the approach to the perfect 
decoding becomes slower near to the critical signal to noise ratio. 

Let us now discuss the "non recursive" models, that is models such that gi(x) divides 
g 2 (x). In this case the energy A(i) to be paid for flipping i]i(er) remains finite in the 
thermodynamic limit. The low X expansions in Eqs. ( |3.3| - |3.4|) have a non vanishing 
term of order O(X). This implies that Q*(X) = 5(X) is no longer a fixed point of Eq. 
(|3.1| ). Let us compute A(i). For "non recursive" models we can define the polynomial 
s(x) = J2k s k xk = 92 {x) / gi (x) . It is easy to show that A(i) = 2Jj + 2 J2k s khi+k- A simple 
approximation of the fixed point of Eq. ( p.l|) is: 

w 

dJ P{J) mdhi P{hi)5 [X - e-W- 2 ^^ (3.22) 

J i=l 

with w = weight (s). This approximation is supposed to be good in the low noise region 
where we expect the distributions Q t {X) to be concentrated on small X's. 
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4 The replica calculation. 



The replica method ]19) starts with the computation of the (integer) moments of the 
partition function. This can be done by introducing an appropriate order parameter (the 
choice is a matter of convenience) and by recurring to standard tricks. Here we choose to 
use the (multi)-overlaps q ai ... ai and their complex conjugates q ai ...a{- 

/at FN FN 

- dq dq / J] - dq a dq a Hj - dq ab dq ab . . . e~ NS ^ (4.1) 

J a 71 J ( a ,6) 71 

S[q,q) = -1 + <?o<?o + ^2q a q a + ^qabqab + ■ ■ ■ + nlog2 + (4.2) 

a (a,b) 
+PJ 7 ld,n[<l) + PFld,n[<i) 

Fu,n[<i\ = - lim — logZi din [g] (4.3) 

N 

Z ld , n [q] = J2U^ + J2^ ei ^ + J2 Qa b e i ((T a )e i (a b ) + ...}■ (4.4) 

{af} i=l a ( a ,b) 



J dV[J]expl-(3j2H(cr a -,J)\ 



where H(cr; J) = — J2i ^ e i( cr ) — Si hirji(<r). and the replica indices a,b, . . . run from 1 
to n. The usual mean field models have no geometrical structure at all. In those cases 
the introduction of the order parameters leads to a (replicated) partition function which 
factorizes over the sites. In our case we are left with the problem of computing the one 
dimensional partition functions Z\^ n \q\. These correspond to the one dimensional sub- 
structures which are not destroyed by the randomness of the model. The saddle point 
equations are easily written 

,. i ^ / 6,K-)..... t .K0 \ 
fa- = & n E W + + < 4 ' 5 ' 

where the expectation values ((•))$ are defined as follows 

/ N 

— — / h,v\j] Vr . ^ 

Z\d. 



((-)>« = ^TT fdV[J]J2(-) f[[<lo + J2 ( l^^ + --^ e '^ aH[(Ta '' J] ( 4 - 6 ) 

d > n ^ J {a?} i=l 



In the recursive case Eq. ( |4.5| ) admits the following solution ^ corresponding to a no-error 
phase: q* ai ^ ai = qti...br = 2^> 2 . The free energy of this phase is fo((3) = — 2 J dJ P{J) J — 
2 j dh P(h) h. If we parametrize the replica symmetric ansatz as in Ref. fl20| 



/+oo 
dxn(x) cosh n (/3x) tanh'(/5x) (4.7) 
-oo 



In fact there is a one parameter family of solutions which are degenerate. This fact is due to a (not 
very interesting) symmetry of the action ( |4.2| ): S[q, q] = S[e l8 q, e~ l8 q\. However the integration over the 
parameter 9 does not pose any problem. We shall fix this freedom by imposing qo to be real. 
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and analogously for b m (with a different distribution 7r(x)), the following free energy- 
functional can is obtained in the limit n — > 0: 

/[tt.tt] = iy"dx^7r(x)^(y)log[2cosh(/5x + ^)]+^/[7r]+^/[7r] (4.8) 



*£V] = ~}^^ JdV[J\]d>K[x\\ogZ*i[J,x] 



Z* s [J,x] = £exp 



A? 



N 



(3 + X i) e *( Cr ) + P X Mi(°0 



i=l 



i=l 



(4.9) 
(4.10) 



The distributions tc and 7r are normalized ( jdx n(x) = jdy 7t(y) = 1) and satisfy the 
saddle point equation below: 

n ( y ) = n^L S / *^ x ] / rfP [ J ] 5 ( y ~ ^ arctanh (( e i(°")> J,aj) + ^ ( 4 - n ) 



which is identical to the fixed point equation corresponding to Eq. (|2.9| ), if we suppose 
the order parameters to be real at the saddle point. 

Equation ( |4.11| ) is unpractical since it involves the unknown distributions tt(x) and 
7r(x) infinitely many times. However due to the short range structure of the hamiltonians 
defined in Eq. Q1.2 ), it can be rewritten as a simple integral equation. Obviously the 
precise form of this equation depends upon the form of the hamiltonian (|1.2p . In particular 
it becomes simpler as the range of the interaction becomes shorter. Let us illustrate this 
point by considering the model [a]] of the previous Section: ej(cr) = <7j<7j_i, T]i(a) = Oj. 
We start by defining the following (right and left) partition functions: 



'i,M 



CTi) 



-1 ...Ci- 



exp 



i+M 



i+M 



exp 



a i _ M ...a i - 1 

and the (right and left) fields: 



(3 {Jk + x k )a k a k -i + (3^2 hk<Jk 

k=i+l k=i 

i i 

+ x k )a k a k ^i + (3 X h k a k 



k= i-M+l 



k=i—M 



(4.12) 
(4.13) 



x 



R/L 



7 {R/L) ( x 

lim rr^ log ■ 



M^oo 2(3 



r(R/L) 
>i,M 1 



(4.14) 



We define now a new couple of order parameters, the probability distributions u(x) and 
lj(x) of the right (or left) fields: 



Y[dhiP(hi) fUdJiPiJi) l'Y[dx i 7T(x i )d(x-x^[J i ;h i ;x i }) (4.15) 

j>0 i>l i>l 



It easy to show that, at the saddle point, u(x) and uj(x) satisfy the following integral 
equation: 



viz) 



dh P(h) J dJ x P(J X ) J dJ 2 P(J 2 ) j dxi Q{x x ) J dx 2 ui(x 2 ) J dz u(z') 
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5 {z - h - [z'; J1 + J2 + Op(xr, x 2 )}} 



(4.16) 



Qp(x; y) = — arctanh[tanh(/3a;) tanh(/3y)] 



(4.17) 



and that the solution of Eq. ( 4.11|) is related to the solution of the previous equation as 
follows: 



7r(x) 



= J dJ P{J) J dxL &{xl) J dx R u)(xr) 5[x — J 



®p(x R ;x L )\ (4.18) 



Equation ( |4.16 ) reduces to the Dyson Schmidt equation |22], ^| for a one dimensional 
model with nearest neighbour interaction if we keep the distribution Cj{x) fixed. The 
interaction between the two one-dimensional subsystems turns it into a nonlinear equation. 
Moreover Eq. ( [4.16 ) can be treated numerically more easily than Eq. ( |4.11| ). A possible 
approach consists in representing the unknown distribution as u(x) = Y2j=i ~ x j) anc ^ 
iterating Eq. ( |4.16| ) until a fixed point is reached. An example of this kind of computations 
is shown in Fig. (0). 

It is simple to obtain the analogous of Eq. ( |4.16j ) for the simplest non recursive model, 
defined by: ej(cr) = <jj, ^(cr) = erjOVi. The final result is 



dh P{h) J dJi P{J X ) J dJ 2 P{J 2 ) J dx l u{x x ) J dx 2 Q(x 2 ) J dz! ou(z') 



S {z - 9/3 [h; J 1 + J 2 + x 1 + x 2 + z'}) 
ir(x) = J dJ P(J) J dx L Cj{x l ) J dx R Cj{x r ) S(x-J-x l - x r ) 



(4.19) 
(4.20) 



A simple approximation to the solution of Eq. ( 4.19 ) can be obtained by starting from a 
distribution u{x) supported on very large fields x and iterating Eq. (|4.19| ) one time. The 
result is 7r(a;) ~ Jd J P(J) J dh x P{hi) J dh x P{hi) 5(x — J — hi — h 2 ), which coincides 
with the more general Eq. ( |3.22| ) after the change of variables X = e~ 2 @ x . No such 
approximation is possible for Eq. ( |4.16|) . 

Expressions equivalent to Eqs. ( 4.16 - 4~T9|) can be derived for more complicated types 
of interaction. In general the distribution lv(x), which is defined on the real line, will be 
replaced by a distribution defined on !R 2r_1 , r being the range of the hamiltonian. 



5 The stability of the frozen solution. 

We would like to study local stability of the no-error phase in the context of the replica 
method. This can be doneQ by computing the eigenvalues of the matrices: 



oqa 1 ...a l oqb 1 ...b„ 



3 For similar calculations see Ref. 24 
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M^g] are the mass matrices for purely real (M + [g]), or purely immaginary (M~[q\), fluc- 
tuations of the order parameter around the value q. We are interested in the saddle point 
Qai a x = 2~ n l 2 . In order to write down all the 2 n eigenvectors of M^g*] it is convenient 
to change slightly our notation for the overlaps. Let us denote by ft C {1, 2, . . . , n} the 
set of I = different indices ( ). We can use the f2's as indices for the overlaps 

with the natural identification = q a n a n. It is not difficult to show that 

(TV) _ 1 d 2 log Z ld>n [q) 



T, 



N dq ai ...a l dq bl . 



(5.2) 



q=q* 



)1— n 



N 



)" 



tanh 



AT2 
~2~ 



J2 dV[J]e nN ^-^ Eo (1 + e 

(id) 

where A(z,j) is defined in Section |3], e" nAW = JdV[J] e ~ nl3H ( (To ' ) and u = ■u ai ...a;,&i...& ro 
counts the indices which are either in the set in Q a = (ai, . . . , aj) or in the set fi& = 
(6i, . . . , 6 m ) but not in both. If q is an eigenvector of with eigenvalue On, then it is 
an eigenvector of M ± [g*] with eigenvalue /x ± = 1 =F limjv^oo @n- 

Notice that is an hermitian matrix with respect to to the scalar product: 



^ y 1ai...ai1ai...ai 
(ai,...,oj) 



E 



(5.3) 



We shall use another subset of {1, 
of T^ N \ which we now exhibit: 



(A) 



n} (let us call it A) to label the different eigenvectors 
1 



2 n/2 



.l)|Ann| 



(5.4) 



The vectors {q^} form an orthonormal set with respect to the scalar product defined in 
Eq. ( |5.3|) . This is easily proven by induction on n. The vector q^ is nothing but the 



constant one. The corresponding eigenvalue is 



(0) 

N 



-1, whence 



^(0) 



2 and 



^(0) 



0. 



The eigenvalue /jlT^ = is a remnant of the invariance of the action under the symmetry 
cited in the footnote |] of the previous Section. In order to compute the eigenvalues in 
the subspace orthogonal to q^ \ the following formula turns out to be useful: 



E 



x 



,|A| (1 



*r' A| ^ A) 



(5.5) 



where QAQ' denotes the symmetric difference of Q and Q' (i.e. QAQ' = (Q\Q')(J(Q'\Q)). 
Using Eq. ( |5.5|) and the results of algebra outlined in Section |] we get (for 0): 



^oo(A) = 2Cj 
where weight (s m ) is defined in Section |3] and 



Cvvi 

m=l 



(5.6) 



(c = (c(\M,n,P) 



J dC P{C) e 
J dC P(C) e 



(n-2\A\)/3C 



(5.7) 
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for C — > h or C — > J. When the one dimensional hamiltonians (|1.2|) satisfy the hypothesis 
of Lemma |3.3| , the sum in Eq. (5.6) can be explicitly computed yielding: 

#iv— oo(A) = ^ \\ (/l j2) (5.8) 

If n > 2|A| then 9(A, n; /3) is positive and decreasing with f3. Moreover lim^oo 9(A, n; (3) = 

and lim^o n; /3) = oo. We can thus define the critical temperatures /3i >n for n/2 > 

1 = | A | > 1 , by requiring [] 

0(A,n;/3, A |, n ) = l (5.9) 

If /3 > /3|A|,n the "frozen" saddle point is stable with respect to the direction g( A ). If 
ft < ft\A\,n it becomes unstable: fit A s = 1 — 9(A) < while /i7 A , = 1 + 9(A) > (it 
could be guessed that the "imaginary" directions would be stable because of the physical 
interpretation of the overlaps). In the limit n — > 0, (3^ n — > f3 c /l\ the critical directions 
are the ones corresponding to |A| = 1. It is easy to see that the critical temperature f3 c 
coincides with the one obtained in Section || 



6 Conclusion. 

We have presented two derivations of the local stability condition for the no-error phase. 
Both will be object of the criticism of the skeptical reader. In the first one we obtained 
the "mean field" equation describing the dynamics of the decoding algorithm, Eq. ( |2.9| ). 
by making use of heuristic arguments. Indeed we argued Eq. (|2.9|) to be valid only in the 
replica symmetric approximation. In the second derivation we made use of the replica 
method, which has not (yet) well founded mathematical basis. 

We think that the two derivations compensate each other for their defects. Moreover 
they yield the same replica symmetric saddle point equation ( |4.11| ) and give the same 
picture of the instability which destroys the no-error (frozen) phase. This corresponds to 
couples of flipped e(cr)'s. Finally thanks to the first derivation we get some insight on the 
behavior of the decoding algorithm. In particular we have seen that, in the frozen phase, 
it approaches a no-error fixed point. This approach becomes slower near to the boundary 
of the frozen phase. 

In Ref. the local stability threshold computed here has been compared with nu- 
merical simulations for two types of code, respectively the models |(a)| and [B] presented 



in Section |3|. Good agreement was found only for model pi) . We propose two possible 
explanations of the disagreement for model |(b)| : 

• the phase transition is a first order one; 

• the turbo decoding algorithm used in Ref. |IJ gets sticked in some local minimum 
of the free energy, characterized by a finite error probability per bit. 

We have not yet enough informations for choosing between these two scenarios. 

61- 



Notice that Eq. (|5.9|) can have more than one solution for n < 2|A|. The "physical" critical point is 



obtained by taking the limit n — > of the solution of Eq. (5.9) which exists for any n 
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A Useful algebra results. 



In this Appendix we remind to the reader some known facts in the theory of finite fields 
and we prove the propositions stated in Section ^. These are nothing but simple exercises 
and we work out them in detail only for greater convenience of the reader. Finally we 
illustrate a few applications of the results obtained. The reader interested in a more 



complete treatment can consult Refs. [25, 26 



Let us begin with some elementary definitions. The basic object is Z 2 i.e. the field of 
integer numbers modulo 2. A polynomial over Z 2 , f(x) G Z 2 [;r] is simply a polynomial 
whose coefficients are in Z 2 . We say f(x) G Z 2 [x] to be irreducible if there do not exist two 
noncostant polynomials g(x), h(x) G Z 2 [x] such that f(x) = g(x) ■ h(x). Any f(x) G Z 2 [;r] 
possess an unique factorization, i.e. a decomposition of the form f(x) = fi(x) Tl -. . .•ff l {x) rh 
where fi(x) G Z 2 [x] are irreducible and > 1 are integer numbers. Given two polynomials 
f(x),g(x) G Z 2 [x] we say that f(x) divides g(x) (in symbols f(x)\g(x)) if there exists 
h(x) G Z 2 [x] such that g(x) = f(x)- h(x) Q For an irreducible polynomial f(x) G Z 2 [x] it 
does make sense to define the order o(f): o(f) is the smallest positive integer k such that 
f(x)\x k + 1. The basic result which we shall employ in this Appendix is the following: 

Theorem A.l Let f(x) be an irreducible polynomial over Z 2 . Then f(x)\x h + 1 if and 
only ifo(f)\k. 

It is useful to know how to compute the order of an irreducible polynomial. The main 
tool is the theorem below: 

Theorem A. 2 Let f(x) be an irreducible polynomial of degree d over Z 2 . Then d is the 
smallest positive integer for which o(f)\2 d — 1. 

Moreover it is obvious from the definition that o(f) > deg(f) 

Our first step will be the proof of Lemma IJO which we restate here as follows 



Lemma A.l Let f(x) be a polynomial on Z 2 with the following factorization 

f{x) = fl\x)-...-f h h {x) ; r,>l (A.l) 

where the polynomials fi(x) are irreducible over Z 2 . Let pi be the smallest integer such 
that 2 Pi > rj. Then f(x)\(l + x k ) if and only if 2 Pi \k and o(fi)\k for i G {1, . . . , h}. 

Proof of Lemma \A.1\ . Let us begin by noticing that, since the fi(x) are irreducible, 
f(x)\(l + x k ) if and only if f[ i (x)\(l + x k ) for i G {1, . . . ,h}. We can then limit ourselves 
to the case f(x) = h r (x) with h(x) irreducible. It is convenient to work in an extension of 
Z 2 , i.e. in a field containing Z 2 as a subfield. We choose an extension (let us call it S) of 
Z 2 such that both h(x) and (1 + x k ) can be decomposed in linear factors. The existence 
of such an extension is a basic fact of field theory. We are then looking for the k such 
that all the root of h(x) (in S) are roots of (1 + x k ) with multiplicity at least r. It is then 
necessary to study the multiplicity of the roots of (1 + x k ). The first observation is that, 

7 Similarly, given two integer numbers p,q £ Z, we say that p divides q (and write p\q) if there exists 
meZ, such that q = mp. 
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if k is odd, all the roots are simple. In fact -r-(l + x ) = kx has no roots in common 
with (1 + x k ). The second observation consists in noticing that (1 + x 2k ) = (1 + x k ) 2 . We 
deduce that (1 + x 2 ™ fc ) with k odd has k distinct roots (the same as (1 + x k )), each one 
with multiplicity 2 m . The final outcome is that h r (x)\(l + x 2mk ) if and only if 2 m > r and 
o(h)\k □ 



From Lemma A.l the explicit form of the period o>(/) used in Sec. || is easily obtained: 

= 2 — lcm(o(A), . . . , (/ A )) (A.2) 



The Lemmas |3.1| and |3.3| are easy consequences of Lemma pT2 . 

Proof of Lemma |ff.4 Let us begin by considering the series l/f(x). We can always 
define the polynomials with deg(y9 n ) < such that lj fix) = Yl^oVnix) x nw ^\ 

Since fix) divides (1 + x muj ^) for all m > 1, we conclude that y5 n (a;) = w(£) = <^(^) 
for all n, n' > and (1 + x muj( -^)/ f(x) = YlT^o v( x ) x ku ^. With the following definition 

L 

g(xMx)^Y,9i^)^ if) , deg[ gi (x)}<u(f) (A.3) 

1=0 

we get 

/x oo min(n,L) oo 

fn = X>™ (/) E <7^) = £* n£J(/) ^) ( A - 4 ) 

n =0 «=0 n=0 

Notice that, for n > L, p n (x) = Poo(x) = g{%)/ f( x ) mod . An upper bound on L is 
easily obtained from Eq. ( |A.3| ) yielding0p„(x) = Poa{x) for n > \(deg[g(x)] — l)/u(f)~\ > 
L. Clearly it cannot be Poo(x) = otherwise we would conclude that f(x) divides g(x) in 
contradiction with the hypothesis. In order to complete the proof of let us suppose the 
following equation to hold 



J>™'K(x) (A.5) 



n=0 



with {J < Lu(f), deg(p^) < to' and p' n (x) = p'^ix) for n large enough. This implies that 
f(x) divides g(x)(l + x w ) but, since gcd(/, g) = 1, we would conclude that f(x) divides 
(1 + x w ) contradicting Lemma □ 



Proof of Lemma 3j . It suffices to specialize the content of the previous paragraph to 
the case deg[g(x)] < ui(f): 



g(x)<p(x) = g (x) + 9l (x) x"W (A.6) 

i \ m—l 
(X) = + X m ^) = g ( X ) + {g Q ( X ) +gi ( X )}J2x h " if) +gi(x) (A.7) 



/l=l 



i Here use the definition \x\ = min{n G Z : n > x}. 
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whence 



weight [s m (x)] = wo + wi ■ m (A. 8) 

wo = weight [g (x)} + weight [gi(x)] - weight [g (x) + gi(x)] (A. 9) 
w x = weight [^o (x) + gi (x)] (A.10) 

□ 

What does it happen when the hypothesis of Lemma EO are not satisfied? It is easy 



to guess the answer. There exists a positive integer m such that, for m > m , weight(s m ) 
grows linearly with m: weight(s m ) = Wo(f,g) + W\(f,g) ■ m with Wi(f,g) = weight(poo). 
Thanks to this fact we can always sum the series in Eq. ( |3.14| ) in the interval < (3 < fli . 



The discussion of the behavior of Eq. ( |3.£| ) presented in Section ^ is then completely 
general. 

Let us return down to the earth and make a few examples. We shall consider the codes 
presented in Ref. 

(a) . The simplest non trivial case: f(x) = 1 +x, g(x) = 1. Clearly both the polynomials 

are irreducible. The degree of f(x) is deg[/(x)] = 1. Because of Theorem |A.2| 
o(/)|2 1 — 1 = 1 whence o(f) = 1 = uj(f). Theorem [A.l| implies that /(x)|l + x k 
for each k > 1. This conclusion is easily confirmed by the well known formula 
(l+x k ) = (l+x)(l+x+. . .+x k ~ 1 ). Lemma |3TT| tells us that g(x)/f(x) = Yl^=oPnX n 
with p n = poo for n > and that p^ = 1 (1 is the unique non zero polynomial of 
degree zero). We have thus rediscovered the simple fact that (1 + x)^ 1 = J2^=o xn - 
Finally we observe the hypothesis of Lemma |3.3| are satisfied and that (with the 
notation of Eq. (|A~6|)), g (x) = 1 and g x {x) = 0. From Eqs. ( EO[ - |OHA~l"0l ) it 
follows that weight [s m (x) = (1 + x m )/(l + x)] = m which is easily confirmed by 
observing that s m (x) — 1 + x + . . . + x m_1 . 

(b) . A less elementary example is: f(x) = 1 + x + x 2 , g(x) = 1 + x 2 . It is easy to 

see that f(x) is irreducible and that g(x) = (1 + x) 2 whence gcd(f,g) = 1. From 
(/)|2 de s(/) - 1 = 3 and o(f) > deg(f) = 2 we deduce that o(f) = 3 = w(f). In fact 

oo 

= i + x + x 3 + x 4 + a; 6 + ... = y^(f( x )x 3n (A.ll) 

1 + X + x 2 

n=0 

(p(x) = l + x (A.12) 

Thus by Lemma ^]2| /(x)|(l + x k ) if and only if k is a multiple of 3. We can use 



Lemma |3T3] in order to compute the weight of s m (x) = (l + x 2 )(l + x 3m )/(l + x + x 2 ). 
We see that go(x) = 1 + x + x 2 and gi(x) = 1 whence weight [/i m (x)] = 2 + 2m. With 
some book-keeping one can confirm this result: 



m—l 



h m (x) =x + ^(x 3m + x 3l+2 ) + x 3m -> weight [h m (x)\ = 2 + 2m (A. 13) 



1=0 



(c). Finally the generating polynomials used in Ref. fU to build the first example of 
turbo code: f(x) = 1+x + x 2 + x 3 + x 4 , g(x) = 1 + x 4 . Also in this case f(x) is 
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irreducible and g(x) = (1+x) 4 yielding gcd(/, g) = 1. Since o(f) |2 deg( ^ — 1 = 15 and 
°{f) > deg(/) = 4, we deduce that either o(f) = 5 or o(f) = 15. However we know 
that (1 + x 5 ) = (1 + x)(l + x + x 2 + x 3 + x 4 ) and we conclude that o(f) = 5 = uj(f). 
In fact 

oo 

= 1 + x + x 5 + x 6 + x 10 + X 11 + ... = Vy?(x)x 5n (A.14) 

1 + X + x 2 ' 

n=0 

^(ar) = 1+x (A.15) 
Using the fact that go(x) = 1 + x + x 4 and gi(x) = 1 we get weight [/i m (x)] = 2m + 2. 



B On the asymptotic behavior of the solutions of Eq. 

In this Appendix we study Eq. ( |3.8|) in order to extend to this case the results concerning 
Eq. (|3.9|) used in Section [3]. We shall examine both the approach of Ref. [13|], which 
is based on the analogy with the KPP equation and is a non rigorous one, and the 
approach of Ref. [ 14] , which employs probability theory and is entirely satisfactory from 



the mathematical point of view. 

We would like to deal with this type of equation: 



/oo poo 00 poo 00 

P{V) dV / Y[p(h) dh / Y[Q n (Zi) dZ t 
■oo J -co i=1 JO i=1 



i-1 



6 [z - E ex p | -p v - p E h i j Zi j ( R 

with the requirement that j dh p(h) h > and the initial condition Pq(Z) = 5(Z — 1). 
Following Ref. [13[ we introduce the function: 



POO 

G n (x) = / dZQ n {Z)exp{-e-? x Z} (B.2) 
Jo 

which satisfy this recurrence equation 

/OO /"OO oo oo % 1 

P(V)dV / Y[p(hi)dhi Y[G n (x + V + Y,h,) (B.3) 
■oo ./-oo i=1 i=1 J=1 

Let us make a few elementary observations concerning Eq. ( |B.3| ): if < G m (x) < 1 for 
some m and all x then < G n {x) < 1 for all x and n > m; if limsup^^ = (700 < 1 

then G n+ i = 0; if G n (a?) is increasing and < G n (x) < 1 for some x (both these hypothesis 
are implied by Eq. (|B.2j )) then lim^^oo G n+ \(x) = 0. The stationary uniform solutions 
of Eq. ( B.3|) are G^(x) = and G^(x) = 1. The first one is obviously stable. If we 
consider a small fluctuation around G^(x), G n (x) = 1 + p n (x) we get: 

/oo poo 00 00 *~ J 

p(v)du / [[p^d/i ^ P „(x + y + J]^) (b.4) 
■00 ./-oo i=1 i=1 i=1 
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which can be diagonalized in Fourier space: 

Pn + l(k) * Y^^yPn(^) = KQPnW (B.5) 

Notice that |A(fc)| > 1 for k small enough and that |A(/c)| diverges at k = in agreement 
with the previous observation that if G n (x) = 1 — p than G n+ i(x) = 0. The preceding 
observations lead us to the hypothesis that the n —>■ oo behavior of our problem is con- 
trolled by front-like solutions G n (x) = g(x — c{(3)n) interpolating between the stable state 
G^(x) = at x — > — oo and G^(x) = 1 at x — > oo. 

This scenario is easily confirmed in the case without disorder. If P(V) = 5(V — Vq) 
and p(h) = 5(h - h ) one obtains P n (Z) = 5(Z - e^ n ), G n (x) = exp{-e~^ x ~ c(/3 ^} 
with 

1 e _/3Vb 

In the general case we assume the existence of front-like solutions with the large x behavior 
G n (x) ~ 1 — e~~^ x ~ c ^ n ^ + o(e~ /3x ). The front velocity is obtained through the construction 
given in Eqs. (^TJ-g]TJ) with 

v(0) = ±hgm = ^og 1 { f{P flh) (B.7) 

Notice that (h) > implies that (e _/3/i ) < 1 in some interval < (3 < (3\ and that 
(3 C < (3\. This remark allows us to sum the series '^2 k (e~ l3h ) k in the range < (3 < j3 c , 
thus obtaining Eq. ( p.7|) . The same remark will be useful in the following. 



Let us consider now the more rigorous approach used in Ref. |L4]. We start by 
defining the polymer model which corresponds to Eq. ( |B.1| ). We have to use a tree with 
a numerable set of branches stemming from each node. A node of the n-th generation is 
identified by n integer numbers u = (u>i, . . . ,u> n ); its generation is denoted by \uj_\. We 
denote by the root node (i.e. the only node of the zeroth generation). We say that 
the node u/ belonging to the m-th generation is a descendant of the node u_ of the n-th 
generation (and write u -< u/ if n < m or u ^ u/ if n < m ) if u)\ = tv[, . . . , u n = 
u' n . The node u/ is said to be an older brother of the node u with \u/\ — \u\ — n if 
Ui = u[, . . . ,u) n -i — ^n-i an d > u' n - A pair of random variables V(co) and h(co) is 
attached at each node. All these variables are statistically independent and have marginal 
distributions p(h) (the /i(cu)'s) and P(V) (the V(u;)'s). A directed polymer is given by a 
pair of nodes uj} -< a; 2 . To each polymer we assign an energy as follows: 

uj}^<uj_-^uP uA^uy^uP 1 ^!'- <*l' ' s an older 

brother of uj_ 

Moreover we use the shorthand E(0, u) — > E(u) and define the following partition func- 
tions: 

ZM= E e ~ m ~ ] (B-9) 

lj: |w[=n 

E e-^^ (B.10) 



CO 



UJ 1 )-~oj: 
\ujf | — \uj_ | — n 
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The velocity of the front wave studied in the previous paragraphs corresponds in this 
language to the random variable: 

c([3) = lim ±-logZ n (/3) (B.ll) 

n->oc nfj 

The model has two phases. In the high temperature phase {(3 < /3 C ) the fluctuations of 
Z n (j3) are small and 

c{(3) = lim \ \og{Z n {(3)) = v{i3) (B.12) 
np 



n^oo 



In the low temperature phase (/3 > j3 c ) the fluctuations become large and c(/3) is fixed 
by simple convexity and monotonicity arguments. The key point of the approach used in 
Ref. JUj is to estimate these fluctuations by proving that, for f3 < (3 C : 

mP<Bou„d(a,« (B.13) 

for some 1 < a < 2 uniformly in n. This is enough for obtaining Eq. ( B.12|) . 

Let us define the normalized variables M n (/3) = Z n ((3) / (Z n (/3)) . In Ref. |14] the 
bound in Eq. (|B.13|) is obtained starting with the second moment of M n (/3), and then 
refining the inequality for the fractional moments of order 1 < a < 2. Notice that looking 
at the ra-th moment of the partition function is a well known method [27] for obtaining 
an upper estimate on the critical temperature (the estimate becomes worser as m gets 
larger). Let us have a look at the first two integer moments: 

oo 

(Z n+ im = (e-^E^Y (ZnW) (B.14) 

A:=0 

(Z 2 n+ M) = ((e- pV )jt(e- ph ) k ) [(Z 2 M) - Z n {2(5)] + 

oo / oo \ 

+(e -2/3V ) J2(e- 2 ? h ) k 1 + 2 YJfi-f*) 1 (Z n (2fl) (B.15) 

fc=0 V 1=1 J 

In general the m-th moment is finite (but not necessarily uniformly bounded) only if 
(e~ m/3/l ) < 1 i.e. if (3 < f3\/m. There is no integer moment of order greater than one 
which remains finite in the interval (0,/3 c ). This fact forces us to a slight modification of 
the proof presented in Ref. Hl4}| . We use the trivial identity: 

Z n+1 ((3)= ]T e-^Z n ((3\u) (B.16) 

ur. |^| = 1 

and estimate the a-th moment (with 1 < a < 2) as follows: 

z: +1 (P) = \ £ E e-^)+^ zm^ 1 ) z n 

uA: uP-: 
U l|=l \^\ = \ 
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< E E e-^^HW)] ^ /2 (/3|^) ^ /2 (/% 2 ) (B.17) 

0,1: u>2: 

l"l|=i \m 2 \=i 

For a temperature such that a/3 < f3\ we can take the averages and sum up the series: 

(Z: + M) < E^ y >™>+ E (e-¥W) + ^)]) (Z a/2 ( ^)2 < 

ljf-l ^^ 2: 

'— 101*1=1 



< 0(«/3)(^(/3)) + 20(«/3)^(e^ ?i ')'^n(/?)) c 



(B.18) 



Rewriting this formula for the normalized variables we get 



(M«(/3))+2 



E( 



e 2 ft ) = 



(B.19) 



At this point we observe, following Ref. fHfl , that, if < (i.e. /3 < /3 C ) then we can 

choose a > 1 such that ip(a, (3) < 1. The condition to be imposed on a for obtaining this 
inequality is a < /3 c /f3 (notice that this inequality implies the previous one a < fii/fl). 
The desired bound is obtained by using Gronwall lemma together with the fact that 
(M «(/3)> = 1: 

(M n Q (/3)) < V>, (3) + \~^} a, ® x(<*, P) < 1 + - ^ ^ X (a, /?) (B.20) 



l--0(«,/9)' 
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Figure V. The dynamics of the turbo decoding algorithm. The graph on the top gives the 
average of the local field r$ (see Eqs.( f2.1| - f2.2| )) as a function of the number of iterations for 
different sizes of the system. The slope of the straight line on the same graph indicates the 
asymptotic velocity obtained in Section |^. The graph on the bottom gives the variance 
of the distribution of the local field. 
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Figure 2: The numerical results for the error probability per bit (stars, *), compared with 
the analytical prediction (continuous line). The analytical prediction is obtained, within 
the replica symmetric approximation, from Eq. ( |4.16j ). This graph refers to model 
defined in Section |[ 
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